Supplementary Material to Kernel Mean Estimation and Stein Effect
نویسندگان
چکیده
Stein’s result has transformed common belief in statistical world that the maximum likelihood estimator, which is in common use for more than a century, is optimal. Charles Stein showed in 1955 that it is possible to uniformly improve the maximum likelihood estimator (MLE) for the Gaussian model in terms of total squared error risk when several parameters are estimated simultaneously from independent normal observations (Stein 1955). James and Stein later proposed a particularly simple estimator which dominates the usual MLE, given that there are more than two parameters (James and Stein 1961). The following proposition gives a general form of the James-Stein estimator.
منابع مشابه
Kernel Mean Estimation via Spectral Filtering: Supplementary Material
This note contains supplementary materials to Kernel Mean Estimation via Spectral Filtering. 1 Proof of Theorem 1 (i) Since μ̌λ = μ̂ λ λ+1 = μ̂P λ+1 , we have ‖μ̌λ − μP‖ = ∥∥∥∥ μ̂P λ+ 1 − μP ∥∥∥∥ ≤ ∥∥∥∥ μ̂P λ+ 1 − μP λ+ 1 ∥∥∥∥+ ∥∥∥∥ μP λ+ 1 − μP ∥∥∥∥ ≤ ‖μ̂P − μP‖+ λ‖μP‖. From [1], we have that ‖μ̂P − μP‖ = OP(n) and therefore the result follows. (ii) Define ∆ := EP‖μ̂P − μP‖ = ∫ k(x,x) dP(x)−‖μP‖ 2 n . ...
متن کاملKernel Mean Estimation and Stein Effect
A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is an important part of many algorithms ranging from kernel principal component analysis to Hilbert-space embedding of distributions. Given a finite sample, an empirical average is the standard estimate for the true kernel mean. We show that this estimator can be improved due to a well-known phenomenon in statistics...
متن کاملTHE COMPARISON OF TWO METHOD NONPARAMETRIC APPROACH ON SMALL AREA ESTIMATION (CASE: APPROACH WITH KERNEL METHODS AND LOCAL POLYNOMIAL REGRESSION)
Small Area estimation is a technique used to estimate parameters of subpopulations with small sample sizes. Small area estimation is needed in obtaining information on a small area, such as sub-district or village. Generally, in some cases, small area estimation uses parametric modeling. But in fact, a lot of models have no linear relationship between the small area average and the covariat...
متن کاملMixture Proportion Estimation via Kernel Embeddings of Distributions
Mixture Proportion Estimation via Kernel Embeddings of Distributions Supplementary Material A. Proof of Propositions 1, 2, 3 and 4 Proposition. d( ) = 0, 8 2 [0, ⇤], b d( ) = 0, 8 2 [0, 1]. Proof. The second equality is obvious and follows from convexity of CS and that both ( b F ) and ( b H) are in CS . The first statement is due to the following. Let 2 [0, ⇤], then we have that, d( ) = inf w2...
متن کاملData-driven Random Fourier Features using Stein Effect
Large-scale kernel approximation is an important problem in machine learning research. Approaches using random Fourier features have become increasingly popular [Rahimi and Recht, 2007], where kernel approximation is treated as empirical mean estimation via Monte Carlo (MC) or Quasi-Monte Carlo (QMC) integration [Yang et al., 2014]. A limitation of the current approaches is that all the feature...
متن کامل